Neural network pruning-the task of reducing the size of a network by removing parameters-has been the subject of a great deal of work in recent years. We provide a meta-analysis of the literature, including an overview of approaches to pruning and consistent findings in the literature. After aggregating results across 81 papers and pruning hundreds of models in controlled conditions, our clearest finding is that the community suffers from a lack of standardized benchmarks and metrics. This deficiency is substantial enough that it is hard to compare pruning techniques to one another or determine how much progress the field has made over the past three decades. To address this situation, we identify issues with current practices, suggest concrete remedies, and introduce ShrinkBench, an open-source framework to facilitate standardized evaluations of pruning methods. We use ShrinkBench to compare various pruning techniques and show that its comprehensive evaluation can prevent common pitfalls when comparing pruning methods.
translated by 谷歌翻译
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.
translated by 谷歌翻译
In this paper, we present an evolved version of the Situational Graphs, which jointly models in a single optimizable factor graph, a SLAM graph, as a set of robot keyframes, containing its associated measurements and robot poses, and a 3D scene graph, as a high-level representation of the environment that encodes its different geometric elements with semantic attributes and the relational information between those elements. Our proposed S-Graphs+ is a novel four-layered factor graph that includes: (1) a keyframes layer with robot pose estimates, (2) a walls layer representing wall surfaces, (3) a rooms layer encompassing sets of wall planes, and (4) a floors layer gathering the rooms within a given floor level. The above graph is optimized in real-time to obtain a robust and accurate estimate of the robot's pose and its map, simultaneously constructing and leveraging the high-level information of the environment. To extract such high-level information, we present novel room and floor segmentation algorithms utilizing the mapped wall planes and free-space clusters. We tested S-Graphs+ on multiple datasets including, simulations of distinct indoor environments, on real datasets captured over several construction sites and office environments, and on a real public dataset of indoor office environments. S-Graphs+ outperforms relevant baselines in the majority of the datasets while extending the robot situational awareness by a four-layered scene model. Moreover, we make the algorithm available as a docker file.
translated by 谷歌翻译
System identification, also known as learning forward models, transfer functions, system dynamics, etc., has a long tradition both in science and engineering in different fields. Particularly, it is a recurring theme in Reinforcement Learning research, where forward models approximate the state transition function of a Markov Decision Process by learning a mapping function from current state and action to the next state. This problem is commonly defined as a Supervised Learning problem in a direct way. This common approach faces several difficulties due to the inherent complexities of the dynamics to learn, for example, delayed effects, high non-linearity, non-stationarity, partial observability and, more important, error accumulation when using bootstrapped predictions (predictions based on past predictions), over large time horizons. Here we explore the use of Reinforcement Learning in this problem. We elaborate on why and how this problem fits naturally and sound as a Reinforcement Learning problem, and present some experimental results that demonstrate RL is a promising technique to solve these kind of problems.
translated by 谷歌翻译
可以使用具有快速有效分割网络的深度学习方法来实施医疗图像分割。单板计算机(SBC)由于内存和处理限制而难以用于训练深网。诸如Google Edge TPU之类的特定硬件使其适合使用复杂的预训练网络进行实时预测。在这项工作中,我们研究了两个SBC的性能,具有和不进行硬件加速度进行底面图像分割,尽管这项研究的结论可以通过其他类型的医学图像的深层神经网络应用于分割。为了测试硬件加速的好处,我们使用先前已发布的工作中的网络和数据集,并通过使用具有超声甲状腺图像的数据集进行测试来概括它们。我们在SBC中测量预测时间,并将其与基于云的TPU系统进行比较。结果表明,使用Edge TPU,机器学习加速SBC的可行性可加速光盘和杯赛分段,每图像可获得低于25毫秒的时间。
translated by 谷歌翻译
通常考虑使用原型生成(PG)方法来提高$ k $ neart nearbor($ k $ nn)分类器的效率。与初始集合相比,这种方法旨在生成降低的语料库版本,而不会降低分类性能。尽管它们在多类方案中进行了庞大的应用,但很少有作品解决了多标签空间的PG方法的建议。在这方面,这项工作介绍了四种多类PG策略对多标签案例的新颖调整。这些建议通过三个基于$ k $ nn的分类器进行评估,其中12个Corpora包括各种域和语料库大小,以及数据中人为诱导的不同噪声场景。获得的结果表明,所提出的适应能够显着改善(在效率和分类性能方面),唯一的参考文献多标记PG在文献中以及没有应用PG方法的情况,也呈现A在嘈杂的场景中,统计上较高的鲁棒性。此外,这些新颖的PG策略允许通过其配置来优先考虑效率或功效标准,具体取决于目标情况,因此涵盖了以前未被其他作品所填写的解决方案空间中的广泛区域。
translated by 谷歌翻译
在计算和数据方面,大型语言模型的预培训通常需要大量资源。经常使用的Web源(例如Common Crawl)可能包含足够的噪声,以使这种预训练的亚地区。在这项工作中,我们尝试了西班牙语版本的MC4的不同采样方法,并提出了一种新颖的以数据为中心的技术,我们将其命名为$ \ textit {Perplexity sampling} $,该技术可实现大约一半的语言模型的预培训步骤并使用五分之一的数据。最终的模型与当前的最新机构相当,甚至可以为某些任务获得更好的结果。我们的工作证明了变形金刚的多功能性,并为小型团队以有限的预算培训模型铺平了道路。我们的型号可在此$ \ href {https://huggingface.co/bertin-project} {url} $中获得。
translated by 谷歌翻译
我们开发了数据驱动的模型,以预测机器人在社交就餐场景中何时应进食。能够与朋友和家人独立饮食被认为是具有行动不便的人的最令人难忘,最重要的活动之一。机器人可以潜在地帮助这项活动,但是由机器人辅助的喂养是一个多方面的问题,在咬合,咬合时机和咬合转移方面面临挑战。特别是在社交就餐场景中,特别是由于在社交用餐场景中变得唯一挑战性,因为可能会中断社交人类机器人群体的互动。我们的关键见解是,考虑到社交线索的微妙平衡的咬合时序策略可能会导致在社交用餐场景中在机器人辅助喂养过程中进行无缝互动。我们通过收集一个包含30组三人共同饮食的多模式人类尊贵数据集(HHCD)来解决这个问题。我们使用此数据集分析人类人类的赋形行为,并在社交用餐场景中开发咬合时正时预测模型。我们还将这些模型转移到人类机器人的态度方案中。我们的用户研究表明,当我们的算法使用食客之间的多模式社交信号线索来建模时,预测会有所改善。 HHCD数据集,用户研究的视频和代码将在接受后公开发布。
translated by 谷歌翻译
我们介绍了一种确定全局特征解耦的方法,并显示其适用于提高数据分析性能的适用性,并开放了新的场所以进行功能传输。我们提出了一种新的形式主义,该形式主义是基于沿特征梯度遵循轨迹来定义对子曼群的转换的。通过这些转换,我们定义了一个归一化,我们证明,它允许解耦可区分的特征。通过将其应用于采样矩,我们获得了用于正骨的准分析溶液,正尾肌肉是峰度的归一化版本,不仅与平均值和方差相关,而且还与偏度相关。我们将此方法应用于原始数据域和过滤器库的输出中,以基于全局描述符的回归和分类问题,与使用经典(未删除)描述符相比,性能得到一致且显着的改进。
translated by 谷歌翻译
移动机器人应该意识到他们的情况,包括对周围环境的深刻理解,以及对自己的状态的估计,成功地做出智能决策并在真实环境中自动执行任务。 3D场景图是一个新兴的研究领域,建议在包含几何,语义和关系/拓扑维度的联合模型中表示环境。尽管3D场景图已经与SLAM技术相结合,以提供机器人的情境理解,但仍需要进一步的研究才能有效地部署它们在板载移动机器人。为此,我们在本文中介绍了一个小说,实时的在线构建情境图(S-Graph),该图在单个优化图中结合在一起,环境的表示与上述三个维度以及机器人姿势一起。我们的方法利用了从3D激光扫描提取的轨道读数和平面表面,以实时构造和优化三层S图,其中包括(1)机器人跟踪层,其中机器人姿势已注册,(2)衡量标准。语义层具有诸如平面壁和(3)我们的新颖拓扑层之类的特征,从而使用高级特征(例如走廊和房间)来限制平面墙。我们的建议不仅证明了机器人姿势估计的最新结果,而且还以度量的环境模型做出了贡献
translated by 谷歌翻译